In Element 1,
we have seen how to subtitle acoustic verbal elements.
In LO1 of this Element,
we have seen how to deal with the visual verbal elements
of an audiovisual text.
In this LO,
we deal with the non-verbal elements of the speech to subtitle,
be they acoustic or visual.
This video lecture is aimed at letting you know
how to identify which non-verbal elements need to be turned into verbal
for each LTA trained working context.
First of all,
I will try to draw on the lessons learnt in LO1,
meaning I will briefly recap
the notions of text, context and semiotics of the audiovisual text.
This will allow me to introduce the question of the communicative impact
of each of the semiotic elements composing an audiovisual text
on the overall communication of the audiovisual text itself.
With this in mind,
I will try to concentrate on non-verbal elements,
both acoustic and visual,
and how to deal with them in the subtitling process
in case they play an important informative role.
This is the agenda of this presentation.
As said, in LO1 of this element
we have seen the features of any text,
which go beyond what is said.
For each of the seven characteristics of a text,
we have given concrete examples to be considered
in real-time intralingual subtitling.
In particular, we have seen the notions of cohesion, coherence,
intentionality, acceptability, informativity,
situationality and intertextuality.
These are rarely considered
if one thinks of respeaking as a mere repetition
of what is said by a speaker.
Then, we have moved to context.
Respeakers not only have to focus on what a speaker says,
but they also need to understand
the relationship of what is said with the context
if they reall want to provide subtitles
that mean something to the target audience.
Indeed,
the meaning-making process of any speech event can only be understood
in relation with its context.
Furthermore, we have briefly analysed the three aspects of any context,
that is its field, tenor, and mode.
Concerning the semiotics of any speech event,
we have finally seen that it communicates
through the combination of many semiotic components.
Be they TV programs, conferences,
parliamentary sessions, meetings or school classes,
they are all of either an acoustic or visual nature,
and they are either verbal or non-verbal.
In particular, we have analysed the different codes characterising
the four semiotic macrocategories of a speech.
Generally speaking, we tend to think
that the verbal elements are more relevant than non-verbal elements
and that acoustic elements are more relevant than visual elements.
However, this is not always the case.
The table in this slide shows the impact on communication
of each semiotic element in the original version of an action movie
and the corresponding impact
of the same movie subtitled for the deaf and the hard of hearing.
Of course, a movie is neither a conference speech,
a parliamentary session, a news item, a school class nor a work meeting.
However, this study shows
that when the subtitler tries to translate the spoken into written,
the attention of the audience is hijacked
by the verbal component of subtitles.
In particular,
the acoustic verbal components and the visual verbal components together
account for only 30% in the original version of a movie
while they account for more than the double, 65%,
in the same movie once it is subtitled.
This happens
because, a deaf or hard-of-hearing person can only rely on the visual channel
to access the acoustic elements of a speech.
In this context,
the live intralingual subtitler turns almost all the verbal acoustic elements
into subtitles,
plus the non-verbal acoustic elements he or she deems essential
to the understanding of the Source Text.
Deciding whether to subtitle non-verbal acoustic elements or not
is not always an easy task.
This normally depends on how they contribute
to the understanding of the event,
or to its unfolding.
If the respeaker decides to subtitle them,
a caption describing the non-verbal acoustic occurrence
is usually enough.
However, the subtitler may not have captions ready at hand.
In preparation of the event to subtitle, we suggest
that the subtitler or the editor drafts a list of possible captions
that could be used while subtitling the event.
As you can see from the examples provided in this slide,
we suggest using expressions that clearly state
something that the audience can immediately understand
like [voting starts]
when the bell announcing the start of the voting sessions rings,
instead of the ambiguous [bell] or [bell ringing].
The same is true for the caption saying [time for assessment is over]
to tell a deaf student
he or she has no more time to dedicate to writing his or her assessment,
instead of [stop]
or [teacher activated alarm clock].
Also, try to avoid general captions like [music]
which means nothing to the eyes of the reader.
Being more specific, like [Intersteno anthem being played],
is for sure more useful and informative.
Finally, try to be as exhaustive as possible
and use more words than you may think are enough.
Instead of [applauses],
prefer [applauses from Lib Dems] if you are subtitling a political speech,
as applauses may generate confusion and make the reader think
the whole parliament approves what is said by the speaker.
What is important to understand here is that,
depending on the setting, such captions are more or less needed.
For example, during conferences their use is limited
to some recurrent things like [applauses from audience]
or [Wim laughs].
During a meeting,
the person receiving personalised access may be more interested knowing
what happens around them.
In the news, it is very common to see
captions translating non-verbal acoustic events,
like [explosion], [ambulance syren], [shots], and similar.
In the case of non-verbal visual elements,
the decision of the respeaker to translate them in the subtitles
also depends on the way subtitles appear on screen.
If they appear as two liners bottom of the screen showing the speaker,
the task is easy,
as the viewer can watch the screen more or less simultaneously
and read the subtitles.
However,
in a conference you may have the speaker projecting a presentation on a screen,
and the subtitles appearing on another one.
In this case the viewer’s experience needs to be facilitated somehow.
Similarly to what happens in dealing with visual verbal elements,
in this case, you may opt for either a momentary transitional caption
telling the audience to watch the other screen
or you can add some words verbalising what images mean.
In the case of a speaker showing a picture and speaking about the picture,
you can either use a transitional caption
like that in the first column which says [watch slide]
or you can verbalise the reference to the picture
and add something like «as you can see from the slide»
to the speaker’s output.
Similarly, when a speaker shows a chart
where several lines indicate how given sectors of a company perform,
it may be useful to inform the audience about what happens in the other screen.
Again, you can either use a caption saying [watch the red line]
or add «the red line indicating a 1% growth»
to the speaker’s output,
if he or she does not mention the piece of data.
Another example may be that of a speaker using a map
to show the position of a city like Milan, without telling it.
The respeaker can either make use of a transitional caption
saying [watch map]
or verbalise the event
by adding «in the north of Italy» to the speaker’s output.
A last case of visual non-verbal element is a new speaker speaking.
This is very common when subtitling classes and meetings,
but also in other contexts like parliamentary sessions,
the news, or the Q&A after a conference.
There are several options to signal a new speaker.
One solution is that of using a name tag,
which is the name of the speaker followed by a colon or a dash
or the name of the speaker into parentheses or brackets.
If you don’t know the name of the speaker
you can simply go to a new line and start the new subtitle with a dash.
In some cases, especially on TV,
turns can be signalled by changing the colour of the text.
Similarly to what happens
with the previous cases of acoustic non-verbal elements
and visual verbal elements,
we suggest you draft a list of these strategies
before the event so as to be ready and use them when needed.
In this video lecture we have summed up
some of the lessons learnt in LO1 of Element 2
and then we have tried to provide
an overview of the strategies to be used in different working contexts
to turn non-verbal elements into subtitles.
In particular, we have recalled the notions of text,
context and semiotics of an audiovisual text,
to then move to the impact non-verbal elements have in an event
and how this impact changes when the same event is subtitled.
Finally, we have seen some of the strategies used to verbalise
both acoustic and visual non-verbal elements.
In the homework session, you will be asked to try and provide solutions
for each situation where non-verbal elements are essential
for the good understanding of a speech.